HDF Overview
Tip: For information on the current HDF version, enter the following at the IDL prompt:
HELP, 'hdf', /DLM
The Hierarchical Data Format (HDF) is a multi-object file format that facilitates the transfer of various types of data between machines and operating systems. HDF is a product of the National Center for Supercomputing Applications (NCSA). HDF is designed to be flexible, portable, self-describing and easily extensible for future enhancements or compatibility with other standard formats. The HDF library contains interfaces for storing and retrieving images and multi-dimensional scientific data.
IDL’s HDF routines all begin with the prefix "HDF_".
Note: For more information on the HDF - EOS tools, please see the "HDF - EOS Tools and Information Center".
Programming Model
Creating HDF Files
The following IDL commands should be used to create a new HDF file:
- HDF_OPEN: Call this procedure first to open an HDF file. To create a new file instead of opening an existing one, set the CREATE keyword.
- HDF_DFAN_ADDFDS: Optionally, add a file description.
- HDF_DFAN_ADDFID: Optionally, add a file annotation.
Adding Data to an HDF File
The routines used to add data to an HDF file vary based on the interface model being used:
- To add an 8-bit image (with or without a palette), use HDF_DFR8_ADDIMAGE or DFR8_PUTIMAGE.
- To add a palette, use HDF_DFP_ADDPAL or HDF_DFP_PUTPAL.
- To add a 24-bit image, use HDF_DF24_ADDIMAGE or HDF_DF24_PUTIMAGE.
- To add a Multi-File Scientific Dataset, use the following commands:
- HDF_SD_CREATE or HDF_SD_SELECT to create an SDS or select an existing one.
- HDF_SD_DIMSET to set dimension information.
- HDF_SD_ATTRSET to set attribute information.
- HDF_SD_SETINFO to insert optional information about the data.
- HDF_SD_ADDDATA to insert the data.
- HDF_SD_SETEXTFILE to move the data to an external file (optional).
- HDF_SD_ENDACCESS to end access to the SDS.
- To add a Vdata, use the following commands:
- HDF_VD_ATTACH to get a Vdata identifier.
- HDF_VD_SETINFO to write information about the Vdata (optional).
- HDF_VD_FDEFINE to prepare non-trivial fields (optional).
- HDF_VD_WRITE to write the Vdata.
- To add a Vdata to a Vgroup, use the following commands:
- HDF_VG_ATTACH to get a Vgroup identifier.
- HDF_VG_SETINFO to set the Vgroup name and class (optional).
- HDF_VG_INSERT to add the Vdata to a Vgroup.
- HDF_VG_DETACH to close the Vgroup.
- HDF_CLOSE to close the file.
Code Examples
Two example files that demonstrate the use of the HDF routines can be found in the examples/doc/sdf subdirectory of the IDL distribution. The file hdf_info.pro prints a summary of basic information about an HDF file. The file hdf_rdwr.pro creates a new HDF file and then reads the information back from that file. Run these example procedures by entering hdf_info or hdf_rdwr at the IDL command prompt or view the files in an IDL Editor window by entering .EDIT hdf_info.pro or .EDIT hdf_rdwr.pro.
HDF Interfaces
There are two basic interfaces to HDF files: the single-file application interface and the multiple-file application interface. These interfaces support eight different types (models) of data access. The table below lists the different models and the names of the IDL routines that access those models. Each model is described in more detail after the table.
|
Model |
IDL Routine Name Prefix |
|---|---|
|
24-bit raster |
HDF_DF24_ |
|
annotation data |
HDF_DFAN_ |
|
palette data |
HDF_DFP_ |
|
8-bit raster |
HDF_DFR8_ |
|
scientific data |
HDF_SD_ |
|
multi-file scientific data |
HDF_SD_ |
|
VData |
HDF_VD_ |
|
VGroup |
HDF_VG_ |
Single File Application Interfaces
In this mode, access is limited to one file at a time. This interface supports the 8-bit raster, 24-bit raster, palette, scientific data, and annotation models. The interfaces are described in more detail after the table.
- 8-bit Raster Model: The HDF_DFR8_ routines access 8-bit images.
- Palette Model: The HDF_DFP_ routines are used to work with the HDF_DFR8_ routines to manipulate palettes associated with 8-bit HDF images.
- 24-bit Raster Model: The HDF_DFR24_ routines access 24-bit images.
- Scientific Data Models (SDs): Used to manipulate arrays of arbitrary dimension and type. Under this model, an array accompanied by a record of its data type, dimensions and descriptors is called a Scientific Dataset (SD).
- Annotation Model: The annotation model is used to describe the contents of the file through such items as labels, data descriptors, and dimension scales.
- Vdata Model: This interface allows for the creation of customized tables. Each table consists of a series of Vdata records whose values are stored in fixed length fields. As described in more detail in the Vdata example below, a Vdata can contain three kinds of identifying information: a Vdata name, Vdata Class, and multiple Vdata field names. The Vdata model is accessed through the routines that begin with the HDF_VD_ prefix.
- Vgroup Model: A collection of one or more data objects, Vdata sets, or Vgroups is known as a Vgroup. Each Vgroup can be given a Vgroup name and Vgroup class. The Vgroup model is accessed through the routines that begin with the HDF_VG_ prefix.
Multi-File Application Interface
The HDF_SD_ routines allow operations on more than one file at a time. This multi-file interoperability is achieved through HDF's use of a modified version of the NetCDF library. IDL’s interface to HDF’s multi-file capability is the HDF_SD_SETEXTFILE routine.
HDF Scientific Dataset ID Numbers
IDL’s HDF_SD_ routines can accept two different types of ID numbers. These ID numbers are referred to in the Help system as the SDinterface_id and SDdataset_id arguments.
The SDinterface_id is the Scientific Dataset interface ID. There is only one SDinterface_id per HDF file. For each actual dataset used, you will also need an SDdataset_id, which is the ID for the particular dataset.
Some routines, such as HDF_SD_ATTRFIND, accept either an SDinterface_id or an SDdataset_id. In these cases, the documentation refers to the ID as an SD_id, meaning that either type of ID is accepted.
IDL and HDF Data Types
HDF and IDL support many different data types. Many of the HDF routines allow you to perform a data type conversion on the fly by setting keywords such as FLOAT. When the data type desired is not explicitly specified, IDL uses the conversions shown in the following tables. Note that single-precision floating-point is the default data type and that the complex data type is not supported.
When writing IDL data to an HDF file, IDL data types are converted to the HDF data types shown in the following table:
|
IDL Data Type |
HDF Data Type |
|---|---|
|
BYTE |
DFNT_UINT8 (IDL bytes are unsigned) |
|
INT |
DFNT_INT16 |
|
UINT |
DFNT_UINT16 |
|
LONG |
DFNT_INT32 |
|
ULONG |
DFNT_UINT32 |
|
FLOAT |
DFNT_FLOAT32 |
|
DOUBLE |
DFNT_DOUBLE |
|
STRING |
DFNT_CHAR8 |
When reading data from an HDF file, HDF data types are converted to the IDL data types shown in the following table:
|
HDF Data Type |
IDL Data Type |
|---|---|
|
DFNT_CHAR8 |
STRING |
|
DFNT_UINT8 |
BYTE |
|
DFNT_INT16 |
INT |
|
DFNT_UINT16 |
UINT |
|
DFNT_INT32 |
LONG |
|
DFNT_UINT32 |
ULONG |
|
DFNT_INT64 |
LONG |
|
DFNT_UINT64 |
ULONG |
|
DFNT_FLOAT32 or DFNT_NONE |
FLOAT |
HDF type codes for the supported HDF data types are shown in the table below:
|
HDF Data Type |
Type Code |
|---|---|
|
DFNT_UCHAR |
3 |
|
DFNT_CHAR |
4 |
|
DFNT_FLOAT32 |
5 |
|
DFNT_FLOAT64 |
6 |
|
DFNT_INT8 |
20 |
|
DFNT_UINT8 |
21 |
|
DFNT_INT16 |
22 |
|
DFNT_UINT16 |
23 |
|
DFNT_INT32 |
24 |
|
DFNT_UINT32 |
25 |
|
DFNT_INT64 |
26 |
|
DFNT_UINT64 |
27 |
Common HDF Tag Numbers
The following table lists common HDF tag numbers and their meanings:
|
Tag Number |
Meaning |
|---|---|
|
030 |
Version Identifier |
|
100 |
File Identifier |
|
101 |
File Description |
|
102 |
Tag Identifier |
|
103 |
Tag Description |
|
104 |
Data Identifier Label |
|
105 |
Data Identifier Annotation |
|
106 |
Number Type |
|
107 |
Machine Type |
|
200 |
Obsolete8-Bit Image Dimensions |
|
201 |
Obsolete8-Bit Palette |
|
202 |
Obsolete8-Bit Raster Image |
|
203 |
Obsolete8-Bit Image-Compressed |
|
204 |
Obsolete8-Bit Image-INCOMP Compressed |
|
300 |
RIG Image Dimension |
|
301 |
Raster Image Look Up Table (LUT) |
|
302 |
Raster Image |
|
303 |
Compressed Raster Image |
|
306 |
Raster Image Group (RIG) |
|
307 |
RIG LUT Dimension |
|
308 |
RIG Matte Dimension |
|
309 |
Raster Image Matte Data |
|
310 |
Raster Image Color Correction |
|
311 |
Raster Image Color Format |
|
312 |
Raster Image Aspect Ratio |
|
400 |
Composite Image Descriptor |
|
500 |
XY Position |
|
602 |
Vector Image - Tek4014 Stream |
|
603 |
Vector Image - Tek4105 Stream |
|
701 |
SD Dimension Record |
|
702 |
SD Data |
|
703 |
SD Scales |
|
704 |
SD Labels |
|
705 |
SD Units |
|
706 |
SD Formats |
|
707 |
SD Max/Min |
|
708 |
SD Coordinates |
|
710 |
SD Link |
|
720 |
SD Descriptor (NDG) |
|
731 |
SD Calibration Information |
|
732 |
SD Fill Value |
|
1962 |
Vdata Description |
|
1963 |
Vdata |
|
1965 |
Vgroup |